Despite its popularity, several recent works question the effectiveness of MAML when test tasks are different from training tasks, thus suggesting various task-conditioned methodology to improve the initialization. Instead of searching for better task-aware initialization, we focus on a complementary factor in MAML framework, inner-loop optimization (or fast adaptation). Consequently, we propose a new weight update rule that greatly enhances the fast adaptation process. Specifically, we introduce a small meta-network that can adaptively generate per-step hyperparameters: learning rate and weight decay coefficients. The experimental results validate that the Adaptive Learning of hyperparameters for Fast Adaptation (ALFA) is the equally important ingredient that was often neglected in the recent few-shot learning approaches. Surprisingly, fast adaptation from random initialization with ALFA can already outperform MAML.

Summary and Contributions: Updated review: After reading the rebuttal, I believe this is a robust empirical paper and worth publishing for the following reasons: - The results are very strong and surprising, challenging emerging hypotheses about generalisation even for image datasets, which are very well studied. This paper convincingly shows that betting on meta-learning adaptation does indeed generalise better than packaging non-adaptive priors via pre-training. This was not at all clear before this paper; indeed, two reviewers needed some convincing to believe it. It's on them that such confusion arose in the first place. Same goes for the title. Priors needed for SOTA generalisation on these well studied datasets can be "packed" into the learning rule itself via meta-learning.

The reviewers generally agreed that this paper brings an important contribution to the NeurIPS community. The results are quite strong, and were surprising to some reviewers. The approach also is scalable. There are multiple changes that we urge the authors to make for the camera ready version of the paper: - The title is far too broad, and not at all informative of the key ideas in the paper. The title should be revised such that it gets across that it is a meta-learning paper, and such that it is specific enough that the new title could not be used to also describe other existing papers.

Despite its popularity, several recent works question the effectiveness of MAML when test tasks are different from training tasks, thus suggesting various task-conditioned methodology to improve the initialization. Instead of searching for better task-aware initialization, we focus on a complementary factor in MAML framework, inner-loop optimization (or fast adaptation). Consequently, we propose a new weight update rule that greatly enhances the fast adaptation process. Specifically, we introduce a small meta-network that can adaptively generate per-step hyperparameters: learning rate and weight decay coefficients. The experimental results validate that the Adaptive Learning of hyperparameters for Fast Adaptation (ALFA) is the equally important ingredient that was often neglected in the recent few-shot learning approaches.

Evolutionary reinforcement learning (ERL) algorithms recently raise attention in tackling complex reinforcement learning (RL) problems due to high parallelism, while they are prone to insufficient exploration or model collapse without carefully tuning hyperparameters (aka meta-parameters). In the paper, we propose a general meta ERL framework via bilevel optimization (BiERL) to jointly update hyperparameters in parallel to training the ERL model within a single agent, which relieves the need for prior domain knowledge or costly optimization procedure before model deployment. We design an elegant meta-level architecture that embeds the inner-level's evolving experience into an informative population representation and introduce a simple and feasible evaluation of the meta-level fitness function to facilitate learning efficiency. We perform extensive experiments in MuJoCo and Box2D tasks to verify that as a general framework, BiERL outperforms various baselines and consistently improves the learning performance for a diversity of ERL algorithms.